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To meet increasing demand, the State University 
System (SUS) of Florida has initiated joint program evaluations with 
the major professional accrediting bodies. As accreditation and 
program reviews typically demand similar information and similar 
procedures the two activities can be synchronized. The purpose of 
assessing and enhancing quality, a cornerstone of both types of 
review, forms the basis for joint reviews. SUS has developed and used 
a variety of models to suit the parameters of each review: first, 
coordinating only the self-study component; second, Board staff 
reviewing the self-study document prepared for accreditation, 
accompanying the accreditation examining team on the site visit, and 
reviewing the accreditation report; third, coordinated site visits 
with different reports; and, fourth, coordination of both site visit 
and self-study. Particularly difficult is synchronizing reviews 
involving programs at several universities. SUS lessons learned from 
coordination efforts include: (1) reviews should be carefully timed 
to allow for preparation and complex scheduling; (2) joint reviews 
should be conducted only with strong accrediting bodies; (3) many 
options should be considered; (4) different models should be used for 
different circumstances; (5) the increased manpower of joint reviews 
can result in greater site visit efficiency; (6) the SUS ccnsultant/s 
should not vote on accreditation standards if involved in the entire 
site visit, (Contains three references.) (JB) 
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Abstract 

Universities routinely undergo evaluations by professional 
accreditation bodies and by governing boards. The increasing 
demands for accountability in education generate inordinate 
pressures for providing large amounts of information and repeated 
scrutiny from multiple stakeholders. To reduce this burden, the 
State University System of Florida, which conducts cyclical 
program reviews of all academic programs within its nine 
institutions, has taken a proactive stance in initiating joint 
program evaluations with the major professional accrediting 
bodies for teacher education, business, education, medicine, and 
pharmacy. This paper discusses the lessons learned from joint 
endeavors and offers suggestions for successful cooperative 
efforts . 
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Joint Accreditation and State Program Reviews: 
Breaking New Ground 

Introduction 

Institutions of higher education routinely undergo multiple 
evaluations each year. These include activities by regional and 
disciplinary accreditation bodies, governing boards, and internal 
self-study committees. The increasing demands for accountability 
in higher education generate inordinate pressures for providing 
increasing amounts of information and repeated scrutiny from 
multiple stakeholders. These demands come at a time when the 
faculty of universities are also being asked or directed to spend 
more time in classrooms. 

In an attempt to reduce the burden of multiple evaluations 
on universities, some evaluation experts have advocated 
cooperative models whereby various entities responsible for 
evaluation synchronize their work (Lincoln, 1990; Barak and 
Breier, 1990.) However, the implementation of such cooperative 
models has been limited. 

The State University System of Florida (SUS) has taken a 
proactive stance in examining, initiating, and implementing joint 
program evaluations involving several of the major professional 
accrediting bodies. The paper discusses the lessons learned from 
the joint endeavors and offers suggestions to institutions of 
higher education and agencies engaged in evaluation for 
successful cooperative evaluation efforts. 
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Accreditation and Program ' Review: Comparisons and Contrasts 
The concept of synchronizing statewide or institutional program 
review and accreditation is based on the premise that there are 
enough similarities between these two types of activities to make 
synchronization viable. Barak and Breier (1990) note the 
following similarities between program review and accreditation: 

a) Both typically require a self-study, an on-site visit, an 
evaluation report, and recommendations. 

b) Both typically use data to assess the program being 
evaluated (p. 118) 

The similarities in procedure and in the types of data assessed 
facilitate the synchronization of the two types of evaluations. 
The goals of the two activities must also be examined, and here 
tviere are both similarities and differences. 

Accreditation usually involves formative reviews intended to 
improve the program, measuring it against predetermined standards 
(Barak and Breier, 1990). Failure to meet the minimum standards 
col ;d result in the failure to gain initial accreditation or to 
maintain accreditation status. 

Program Review may be either formative or summative 
evaluation, which may involve not only program improvement but 
also program termination. It goes beyond concern with meeting 
minimum criteria and may involve an interest in increasing 
efficiency. The link between program review and cost 
effectiveness, and the additional dimensions of program 
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termination and moving beyond minimum predetermined standards are 
important differences between program review and accreditation, 

A recent dissertation study conducted by Moon-Hee Lee (1991) 
provides information regarding the purposes of program review in 
the 39 states which conduct such reviews on a statewide basis, 
and the purposes of the regional accrediting agencies. The 
purposes of regional accreditation differ somewhat from the 
purposes of program accreditation, but there are at least two 
purposes held in common: 1) Quality — assessing and enhancing 
educational quality, and 2) Assurance — assuring constituencies 
that the institution or the program meets minimum quality 
standards . 

A comparison of the purposes of accreditation and program 
review provides useful insight to those interested in 
s ynchronizing the two types of evaluative activities. The most 
frequently identified purposes of program review were: 

♦ Assessing and enhancing program quality (62%) 

♦ Increasing responsiveness to changing program need (49%) 

♦ Increasing efficiency and effectiveness of resources (36%) 

♦ Consumer protection — providing optimum service and access (23%) 

♦ Facilitating statewide planning and budgeting (23%) 

♦ Eliminating costly duplication of programs (23%) (Lee, 1991), 

The purpose of assessing and enhancing quality, a 
cornerstone of both accreditation and the majority of program 
reviews, forms the basis for joint reviews. Two other program 
review purposes mentioned above also relate to accreditation. 
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Increasing responsiveness of a program to changing program 

need may be part of accreditation in the sense of examining 
programmatic responses to changes in the discipline. The purpose 
of consumer protection in program review usually involves 
protection through optimum service and access to constituents, as 
well as protection through acceptable quality of offerings . 
While the issues of service and access may not apply to 
accreditation, the issue of consumer protection through 
certification of meeting minimum standards is certainly central. 

When one moves from the purposes of program review and 
accreditation down to the more specific level of considering the 
criteria involved in the reviews, clear areas of similarities and 
divergences between program review and accreditation begin to 
emerge. The most frequently utilized criteria in state-level 
program reviews, as noted by Lee (1991), and the authors 1 
assessment of whether they are generally considered in 
accreditation reviews is indicated in the following table: 



Program Review Criteria 


Applicable to Accreditation 


Program quality 


Yes 


Cost effectiveness of academic 
program 


No, but provision of adequate 
resources is an issue 


Student demand 


No 


Program productivity 


Inferred, but not central 
consideration 



Table Continued 
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Program Review criteria 


Applicable to Accreditation 


Centrality of program to 
institutional mission 


Yes 


Program duplication 


No 


Statewide manpower need 


No 


Interconnectedness of a 
program with other programs 


Contribution of other programs 
to program reviewed 


Geographical access to program 


No 


Student outcomes 


Yes 



Although on its face the table seems to indicate several 
areas of divergence in criteria between program review and 
accreditation, most data elements and other information used to 
evaluate the criteria and, therefore, included in self-studies in 
program review, are generally included in accreditation self- 
studies as well. Some of the criteria identified above as being 
unique to program review, such as state manpower needs and 
geographical access to program, are typically obtained from 
sources other than the self-studies ♦ 

Types of Joint Reviews 

Joint reviews may take several forms, and the SUS has 
developed and utilized a variety of models to suit the specific 
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parameters of each review. 

One of the easiest types of coordination to accomplish is 
coordination involving self-studies alone. In such cases the SUS 
examines the self -study format or actual self -study prepared for 
accreditation in the discipline to be reviewed, determines what 
additional information is needed for purposes of the state 
review, and requests that information as an addendum to the 
accreditation self-study. This approach is currently being used 
in an SUS review of Communication programs, using self-studies 
prepared for the Accrediting Council on Education in Journalism 
and Mass Communications (ACEJMC) accreditation. 

Such coordination can occur when the timing of the state 
review is reasonably close to that of the accreditation review. 
It has the advantage of being a simple means of drastically 
inducing the effort required by the universities to develop two 
self-study documents. This coordination, however, does not 
eliminate the need for separate on-site visits. 

Another relatively simple model of joint review involves 
staff from the Board reviewing the self-^tudy document prepared 
for accreditation, accompanying the examining team for 
accreditation on the site visit, and reviewing the accreditation 
report. This model involves obtaining agreement by the 
accrediting agency and/or the university to join in the site 
visit. While this model eliminates the need for a separate state 
review of the program, it has the disadvantage of not addressing 
the issues which are of interest to the state but not to the 
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accrediting agency. 

Some programs, such as those in Nursing, undergo review not 
only by an accrediting agency but also by a state licensing 
board. Coordinating reviews with the state licensing board, 
which may have a less arduous review process than accreditation, 
can be accomplished with relative ease depending on how 
cooperative the licensing board is. The SUS conducted a joint 
review of several nursing programs with the Florida Board of 
Nursing (BON) . The SUS and the BON each required a separate 
self -study, because the format was sufficiently different to make 
coordination of the self -studies cumbersome. The site visits 
were coordinated, but the reports were separate. 

Coordinating both the self -study and the site visit is the 
most complete type of synchronized review. This model involves 
t;ie most effort in coordination, but that effort is offset by the 
reduced effort on the part of the university and the program in 
preparing for just one self-study and visit. The SUS initiated 
and participated in the first tripartite joint review in the 
United States involving the National Council for Accreditation of 
Teacher Education (NCATE) , the State Department of Education 
(DOE) , and the State University System. The SUS has also 
negotiated with the American Assembly of Collegiate Schools of 
Business (AACSB) to conduct a completely synchronized review in 
1997. 

The main difficulty in conducting joint reviews arises in 
synchronizing reviews involving programs ar several universities. 
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The SUS, for exarc le, is comprised of nine universities. 
Programs which are offered at only one or two universities (such 
as medicine, dentistry, and pharmacy) are relatively easy to 
coordinate with accreditation reviews. However, as the number of 
universities offering a particular discipline increases (such as 
in Education and in Business), synchronization becomes more 
complex, particularly if the state wants to have e, systemwide 
perspective of all programs within a limited time period. 
Programs at various universities in a system do not typically 
undergo accreditation at the same time. Therefore program 
reviews which are scheduled to coincide with accreditation could 
string out a program review of a particular discipline over a 
period of five to seven years. 

This is certainly not conducive to obtaining a statewide 
perspective of the status of a discipline in the system. 
Compressing synchronized reviews into a one- or two-year time 
span requires considerable accommodation from the accrediting 
agency and the universities involved. The SUS has been fortunate 
to work out such a commitment from AACSB, and it is for this 
reason that the joint review holds particularly high promise. 

Coordination of Self-Studies 

The experience of the SUS with seven accrediting agencies in 
the last two years indicates that the type of information 
requested in the program review self-study is also requested in 
the accreditation self-studies in the major areas of curriculum, 
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student admissions and retention, student services, faculty 
teaching and research, program budget, library resources, 
laboratories, space, and support staff. The areas of interest to 
the state which were usually not included in accreditation self- 
studies related to accountability and cost-effectiveness, access, 
student demand, articulation with community colleges, feedback 
from graduates, degree productivity, new program development, and 
the success of particular enhancements such as the Florida's 
Eminent Scholars program. Equal opportunity/ af f irmative action 
information which is requested in the state self-studies is 
beginning to be requested in some accreditation self-studies as 
well. In terms of specific data forms required in the state 
program review s^lf-studies, the following table indicates, in 
each case, the number of accrediting agencies (from a total of 
i3ven) which also require similar information in their self- 
studies. 



Program Review Self-Study 
Forms 


Number of Accreditation Self- 
Studies including Similar 
Information 


Number of majors, including 
transfers 


3 (no information on 
transfers) 


Admission and enrollment 


4 


Graduate placement 


3 


Faculty profile 


7 
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Faculty budget 


5 


Support personnel 


4 


Required courses 


7 


Special cost courses 


0 


Internship information 


5 


Contracts & Grants 


3 


Space allocation and needs 


6 


Equipment budget and needs 


3 


Results of other program 
evaluations 


0 


Library collection 

* 


7 


Summary faculty vita 


2 (full vita) 


Student FTE productivity 


0 


Breakdown of course rovpranp 
by regular faculty, adjuncts, 
and teaching assistants 


0 f 1 t npl hHdH 
u ^ocvciai lllLXUUcu 

information on numbers of 
part-time faculty) 



The few specific items required in the SUS self -study 
documents but not in accreditation self-studies relate to matters 
of interest to the State legislature, such as access, proportion 
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of courses taught by adjuncts and teaching assistants, and FTE 
productivity. Other states probably have their own sets of items 
which are of particular interest to the state but are not part of 
an accreditation review. 

Advantages, Disadvantages , and Lessons Learned 

This section will summarize a few of the more important 
advantages and disadvantages of joint review activity, and offer 
advice for those interested in pursuing the possibilities of 
joint review. 

1. Timing is Important. This is true for two reasons. 
First, the time involved with preparing for a successful joint 
review can depend upon the complexity of the model, but, in any 
case, the more time that is spent in planning increases the 
likelihood of success. For example, the SUS intends to conduct 
;>int reviews involving single site visits with accrediting 
bodies in pharmacy and in dentistry at the SUS institutions where 
degrees are offered, but these amount to only two universities. 
Therefore, the model, the necessary Memorandum of Understanding, 
and the addendum to the accrediting body can be created within a 
year. Planning for a joint review with the AACSB, however, began 
in 1991 with a target date of 1997. This was necessary because 
(a) nine institutions will be involved, and (b) the site visits 
needed to be collapsed into a single year; hence, a year well 
into the future had to be assumed and then negotiated both with 
the AACSB and the universities. 

Secondly, the calendar of site visits is of consideration if 
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several universities are involved. As indicated, the joint 
review to be conducted with AACSB was especially attractive 
because of the accrediting body's willingness to collapse all 
activities into one year. The tripartite review of teacher 
education (NCATE, SUS, and DOE) operates under the NCATE 
calendar. As a consequence, the program review is virtually 
ongoing from the SUS standpoint. This makes SUS reporting 
difficult, and it mandates that the SUS maintain staff to work on 
the review. 

2. Strength is Important. There is no point in working 
toward joint reviews with accrediting bodies who are not viewed 
as extremely strong by the academic disciplines. 

3. Consider Options. The LCME pointedly did not wish to 
engage in joint review activities with the SUS. The SUS, 
subsequent to this understanding, went directly to its two 
affected universities, and inquired whether they preferred to 
undergo one review instead of two. The universities immediately 
provided assurance that the SUS would receive institutional self- 
studies, invitation to the presidential exit interviews, and the 
accrediting body's report. 

4. Different Models for Different Circumstances. The 
agreement reached between the two SUS universities and the SUS 
with respect to LCME accreditation points out that those wishing 
to work toward multiple reviews should expect variations on 
models, subject to whether or not a level of comfort is attained 
that programs will receive sufficient scrutiny. The joint review 
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in Medicine, for example, resulted in and amounted to little real 
interaction between the SUS and the accrediting body; even so, 
the SUS was comfortable with the strength of that accrediting 
body and the level of scrutiny, 

5. Doubling the Manpower. One of the most evident 
advantages of joint reviews is that it provides for more manpower 
during the site visit. This is less advantageous in cases where 
a single SUS consultant participates with an accreditation review 
team. However, in the case of the tripartite review of teacher 
education, the manpower on site is doubled due to the presence of 
three agencies at once. This has increased efficiency as well as 
confidence that institutions are receiving adequate scrutiny. 

6. To Vote or Not to Vote. It is the position of the SUS 
that it not be afforded opportunities to vote on accreditation 
standards in those joint review cases where it has employed a 
consultant who participates in the entire site visit. Such is 
the case with the tripartite NCATE, SUS, DOE joint review of 
teacher education: NCATE votes on state program approval 
standards with members of the DOE team. Likewise, the DOE team 
members vote on NCATE standards with the NCATE team. The SUS 
consultant, while an active participant of the process, is not a 
voting member. 

In order to conduct successful joint reviews both the 
similarities and differences between the particular program 
review model utilized and the accreditation process must be 
carefully examined prior to each joint review. Where important 
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differences exist, the willingness of the accrediting agency to 
accommodate program review, and the willingness of program review 
coordinators to compromise, are key in conducting a review that 
meets the needs of both parties adequately, while resulting in a 
significant savings in cost, time and effort for the program 
being evaluated. 



14 



18 



REFERENCES 



Barak, R. J. and Breier, B. E. Successful Program Review , San 
Francisco: Jossey-Bass Publishers , 1990 . 

Lee, M. A comparative analysis of academic program review by 
state governing boards, state coordinating boards, and regional 
accrediting agencies in American higher education, Ed.D. 
dissertation, Charlottesville, VA: Curry School of Education, 
University of Virginia, 1991. 

Lincoln, Y. S. "Program Review, Accreditation Processes, and 
Outcomes Assessment. 11 Evaluation Practice 11 (1) , 1990. 

1 



15 



IB 



